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Nowadays, the number of road accident in Malaysia is increasing 
expeditiously. One of the ways to reduce the number of road accident is 
through the development of the advanced driving assistance system (ADAS) 
by professional engineers. Several ADAS system has been proposed by 
taking into consideration the delay tolerance and the accuracy of the system 
itself. In this work, a traffic sign recognition system has been developed to 
increase the safety of the road users by installing the system inside the car 
for driver’s awareness. TensorFlow algorithm has been considered in this 
work for object recognition through machine learning due to its high 
accuracy. The algorithm is embedded in the Raspberry Pi 3 for processing 
and analysis to detect the traffic sign from the real-time video recording 
from Raspberry Pi camera NoIR. This work aims to study the accuracy, delay 
and reliability of the developed system using a Raspberry Pi 3 processor 
considering several scenarios related to the state of the environment and the 
condition of the traffic signs. A real-time testbed implementation has been 
conducted considering twenty different traffic signs and the results show that 
the system has more than 90% accuracy and is reliable with an acceptable 
delay. 
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1. INTRODUCTION 


Nowadays, the number of road accident in Malaysia is increasing expeditiously. As reported in [1], 
Malaysian has the third highest fatality rate of road accidents in ASEAN, with 7,512 Malaysian citizens’ 
death in 2016. It also reported that the fourth most common cause of death for Malaysian people is road 
accidents. Traffic sign recognition system is a pivotal platform that can be implemented in the autonomous 
driving system. Sometimes, the driver may overlook the traffic sign due to tiredness, and this will indirectly 
cause road accidents [2]. Besides, the blockage of the traffic signs by big vehicle and tree also cause the 
driver to overlook the traffic sign. Hence, a traffic signs recognition system that is able to detect the traffic 
signs is necessary to alert the driver. The development of the traffic sign recognition system has been 
considered in the past few years to reduce the number of accidents. The first traffic sign detection and 
recognition (TSDR) system was developed in Japan since 1984 [3]. The TSDR system used computer vision 
with eight standard datasets recognition techniques, included German TSDR benchmark (GTSDRB), KUL 
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Belgium traffic sign dataset (KULD), Swedish traffic signs dataset (STSD), The Netherlands RUG traffic 
signs database (RUGD), France Stereopolis database, United States LISA dataset (LISAD), United Kingdom 


online dataset (UKOD) and Russian traffic sign dataset (RTSD) for fast detection of the traffic signs. 
Figure 1 shows a sample of traffic signs. 


TORQ ae® 
BEHOP EEO. 


oo 
0 E- BNGOLE 


Figure 1. Dataset sample of traffic signs [3] 


Advanced driving assistance system (ADAS) is a system developed by professional engineers with 
the aims to reduce the number of road accidents. This include an intelligent system to detect vehicles for 
traffic management purposes [4]-[6] and the traffic sign along the road [7]. However, the type of machine 
learning algorithm used in the traffic sign recognition system to efficiently detect the traffic sign is very 
important. The weather condition and the reflection of light on the traffic sign can reduce the quality of the 
captured traffic sign image [7]. Therefore, an efficient algorithm is required to perform the processing and 
analysis of the captured image. Several machine learning algorithm has been proposed for traffic sign 
recognition include the TensorFlow transfer learning algorithm [8]-[10], AdaBoost algorithm [11], 
convolutional neural networks (CCN) algorithm [12], [13], fuzzy integral algorithm [14], neural network 
[15], artificial neural network (ANN) [16], deep learning [17], [18], color transformation [19], and texture 
feature extraction [20]. The types of algorithms used inside TensorFlow, such as transfer learning, increase 
the efficiency of the traffic sign recognition when compared to traditional machine learning. Note that 
transfer learning is developed using information or knowledge from the surrounding environment [8]. The 
training and test samples are randomly selected, which can be a large amount of data or a small amount of 
labelled data. Transfer learning allows training only at the last layer of the whole Inception-v3 network, and 
this can help to reduce the training time and maintain the accuracy of recognition [21]. AdaBoost algorithm 
has been proposed to reduce the noise of the segmented traffic sign images. To increase the accuracy, the 
AdaBoost algorithm requires a big dataset of sample images to detect the traffic sign. According to [22], a 
CNN ensemble is used in the traffic sign smart detection system, which focuses on colour-based detection, 
shape-based detection and sign validation. According to [23], CNN can be used to annotate the sample 
images as CNN is a class of models that is easier to be trained with the sample images. Besides, a data 
segmentation method is also used in CCN to reduce the image data overfitting by performing the process of 
label-preserving transformation. However, the colour of the traffic sign and the lack of light during night may 
effected the performance of the CCN. 

Meanwhile, several researchers considered Raspberry Pi, which has limited computation capability 
to develop the IoT-based application system [24]-[27]. As in [24], the authors used the Raspberry Pi to 
observe wildlife using deep learning, giving up to 97% accuracy. Akshay and Dinesh in [25] study the 
shading effect on the road sign recognition system that runs on the Raspberry Pi. However, the performance 
of the system is not clearly reported. Kharkar [26] developed a road sign detection and recognition robot 
system using an edge detection algorithm running on the Raspberry Pi to detect the traffic sign. However, the 
performance of the developed system is not evaluated in a real environment which might introduce a 
significant impact (i.e. reflection of light) on the captured traffic sign image. Meanwhile, the authors in [27] 
developed a road sign recognition system using Raspberry Pi but only focused on the speed sign while 
considering the stability of colour detection due to daylight condition. The results show that the system has 
80% accuracy with a processing delay of up to 2 seconds. 
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In this work, a real-time traffic sign detection and recognition system using the TensorFlow 
machine-learning algorithm to process and analyze the sample model of traffic signs has been developed to 
assist the driver. The system used Raspberry Pi 3 that is embedded with the TensorFlow algorithm, to detect 
the type of traffic sign based on the real-time video recording obtained by Raspberry Pi camera NoIR and 
send alerts to the driver. The accuracy, the reliability and the delay of the developed system are evaluated and 
analyzed through a real-time testbed implementation in a real environment considering five different class of 
traffic signs, the condition of the traffic sign itself (i.e. blurred, and broken) and also the state of the environment. 


2. RESEARCH METHOD 

Raspberry Pi is a mini-computer that is capable of running applications as the same standard desktop 
computer [28]. In this work, a Raspberry Pi 3 module is used to run the TensorFlow machine learning 
algorithm and is connected to a Raspberry Pi camera NoIR to record the real-time video, a Raspberry Pi 
Display module to display the related information of the detected traffic sign and a speaker to give an alert 
sound to alert the driver. Figure 2 shows the complete hardware of the developed system. Note that the 
developed system will be deployed in a car on the dashboard and is powered using power car adapter. 


(a) 


Figure 2. The developed real-time traffic sign detection and recognition system; (a) front view, (b) side view, 
and (c) rear view 


Meanwhile, the working principles of the developed real-time traffic sign recognition system are as 
shown in Figure 3. First, the Raspberry Pi 3 and Raspberry Pi camera NoIR are in stand-by mode once the 
system is powered on. Then, the Raspberry Pi camera NoIR will start records the real-time video and share 
the video with the Raspberry Pi 3 synchronously. The Raspberry Pi 3 will process the video to detect the type 
of traffic sign. If a traffic sign is detected, the results will be sent to the Raspberry Pi Display Module to 
display the type of the detected traffic sign and activate the speaker. Otherwise, the camera will keep on 
recording and sending the data to the Raspberry Pi 3 module for processing until the traffic sign detected. 


2.1. Labelimg 

Labelimg is also known as label image, which is an open-source image labelling tool that is used to 
label the size of the traffic sign image dataset. Each traffic sign image needs to be labelled before starting the 
training process of the traffic sign images with TensorFlow machine learning software. The purpose of 
labelling the traffic sign images is to determine the location of the traffic signs. Besides, Labelimg carries out 
the process of segmentation of traffic sign images and then continues with the process of annotation and 
interpretation for the traffic sign images. Annotation of traffic sign images is important as the bounding 
outside boxes will be shown on the traffic sign images, thus the images will be easily recognized, as shown in 
Figure 4. The coding used for Labelimg is Python programming language. After Labelimg annotates the 
sample set of traffic sign images, it saves the images as ‘XML’ file format, and the images are ready to be 
trained and tested by TensorFlow. 


2.2. TensorFlow machine learning 

As stated above, in this work, the TensorFlow machine learning algorithm is used to train a dataset 
consisting of five different classes of traffic signs, including the Speed Bump, Stop, Give Way, No U-Turn 
and Chevron Alignment. These five classes of traffic signs are considered as they are commonly found at the 
roadside [29]. For each class, there are 100 sample images with different angles, and size hence give a total 
of 500 sample images. The training process is important to prepare the pre-trained model before the 
implementation of the real-time recognition system. The pre-trained model that contained the size, colour, 
shape and boundary of the sample traffic sign images is used for the recognition. The flowchart of the 
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TensorFlow algorithm is as shown in Figure 5. In this traffic sign recognition system, TensorFlow Lite 
version 2.5.0 machine learning software is installed on the Raspberry Pi 3 to run the pre-trained model, which 
consists of the dataset that was trained before. For the traffic sign recognition, the Raspberry Pi 3 will run the 
TensorFlow algorithm to detect the traffic signs based on the real-time video recording from the Raspberry Pi 
camera NoIR with the help of the Labelimg results performed for traffic sign classification as discussed in 
section 2.1. Once the traffic sign is detected, the Raspberry Pi Display Module will show the type of that 
traffic sign with the percentage of its accuracy. Figure 6 shows the coding to run TensorFlow software on 
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Raspberry Pi Camera NoIR module 
records the real-time video 
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Figure 3. The flow system of the real-time traffic sign detection and recognition system algorithm 
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Figure 4. Labelling the traffic sign sample image 
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Import sample images 
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Label the image using Labelimg 
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Train model by size, colour, shape, and boundary 
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Figure 5. Flowchart of TensorFlow machine learning image classification 
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Figure 6. Coding for starting detection using TensorFlow machine learning 


3. RESULTS AND DISCUSSION 

In this work, the performance of the developed real-time traffic sign detection and recognition 
system in terms of accuracy, reliability and delay has been evaluated considering five different classes of 
traffic signs, the state of the environment and the condition of the traffic sign. The developed system is 
installed in a car, and the speed of the car considered in this experiment is 50 kilometres per hour (50 km/h). 
Note that the locations considered to perform the testbed implementation of the developed system are at 
Taman Muzaffar Height, Ayer Keroh and Kawasan Perindustrian Ayer Keroh, in which the five classes of 
traffic signs considered in this work are available. Figure 7 shows the samples images that are displayed on 
the Raspberry Pi display module when the traffic sign is detected during the experiment. Note that the 
percentage of the accuracy of the detected traffic sign is also displayed on the Raspberry Pi display module. 
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Figure 7. Screenshot of (a) speed bump, (b) stop, and (c) give way, traffic sign is recognized from the front view 
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3.1. System accuracy 

To evaluate the accuracy of the developed system, 20 traffic sign images from the five different 
classes of traffic signs are captured during the testbed implementation. Note that the testbed implementation 
is performed during normal weather condition at a speed car of 50 km/h. Figure 8 shows the average 
percentage of the accuracy of the five classes of the traffic sign. The results show that the average percentage 
of the accuracy for all classes of traffic sign image is above 90%. This indicates that the developed system works 
efficiently to detect the traffic sign images and identify the type of traffic sign based on real-time video recording. 


3.2. Delay 

The performance of the developed system in term of delay (i.e. processing time) is also evaluated for 
the 20 traffic sign images from the five different classes of traffic sign captured during the testbed 
implementation. The delay refers to the time taken to load and read the pre-trained model when the real-time 
video is running. Note that a normal speed of the car, which is 50 km/h in normal weather, is considered 
during the testbed implementation. Figure 9 shows the average time delay in the developed system to detect 
and identify the type of traffic sign images from the five traffic sign classes. The results show that the 
maximum average delay is 3.44 seconds while the minimum is 3.23 seconds. This shows that the developed 
recognition system is suitable for the city speed limit of the car. However, it is not recommended for a fast- 
moving car (i.e. above 50 km/h) to avoid the latency in the system, hence endanger the driver. 
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Figure 8. Percentage of accuracy for different classes of traffic sign 
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Figure 9. Delay for different classes of traffic sign 


3.3. Reliability 

The reliability of the developed real-time traffic sign recognition system is evaluated considering the 
broken traffic sign such as the incomplete part of the traffic sign or the faded traffic sign and the lighting at 
the testbed implementation (i.e. at night-time). Figure 10(a)-(c) show the broken traffic sign image with its 
detected percentage of accuracy, the detected traffic sign images during night-time with its percentage of 
accuracy, and the considered faded traffic sign with its percentage of accuracy, respectively. For all 
scenarios, the results show that the developed system able to perform well with an accuracy of the traffic sign 
signal above 90%. 
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Figure 10. The traffic sign detection considering; (a) incomplete traffic sign, (b) night-time condition, and 
(c) faded traffic sign with the percentage of accuracy 


4. CONCLUSION 

In this work, a real-time alert system for traffic sign recognition system is developed using the 
Raspberry Pi 3 processor. The TensorFlow machine learning algorithm is used to train the traffic sign and 
detect the traffic sign through the real-time recording video using Raspberry Pi camera NoIR to alert the 
drivers. Twenty traffic sign images from five different classes of traffic sign have been considered for the 
real-time testbed implementation. The performance of the developed system has been evaluated in terms of 
accuracy, delay and reliability. The results show that the average accuracy of detecting and identifying traffic 
sign images from the five traffic sign classes is above 90%. 

Meanwhile, the results also show that the maximum average delay in determining the type of traffic 
sign in the system is 3.44 seconds when the car moves at 50 km/h. This shows that the system is suitable to 
be used for a city speed limit of the car. In term of reliability, the developed system is tested considering the 
condition of the traffic sign such as broken and blurred traffic sign and the state of the environment (i.e. 
during night-time). The results indicate that the developed system is reliable to detect the type of traffic sign 
with 90% accuracy for all considered conditions and send out an alert to the drivers. 
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